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Abstract. Authors of this paper carried out a broader international research aimed at assessing 
the computer science education at upper secondary level of education - ISCED 3A. The assessed 
school subjects were informatics and programming as the most common school subjects taught 
at secondary schools within computer sciences. The assessment was based on the students’ evalu¬ 
ations, their points of view and opinions, what was a specific feature of the research. The paper 
presents main findings and results for the school subject programming obtained in Slovakia (SK) 
and the Czech Republic (CZ). As a weakness of teaching programming in both countries inade¬ 
quate quality of textbooks or a lack of appropriate textbooks in general was identified. As strength 
of the programming teaching in the Czech Republic attractiveness of the content of the school 
subject programming (its curriculum) was identified and in Slovakia it was the clearness of teach¬ 
ers’ presentations of teaching material. 

Keywords: teaching programming, grammar school, quality of teaching, factors influencing the 
quality of the education process, strengths and weaknesses of programming teaching. 


1. Introduction 

During the period of the years 2010-2012 we carried out a broad research aimed at as¬ 
sessing the computer science education at upper secondary level of education (ISCED 
3A) in Slovakia (SK), Czech Republic (CZ) and Belgium (BE). Due to a variety of 
school subjects related to computer science taught at the secondary schools we limited 
our attention only to teaching informatics and programming which are two basic school 
subjects relevant to this area. Due to the diversity of focus in secondary school level and 
a wide range of specific implementation of computer science education in these schools, 
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we focused our attention on a single type of school in this category: in Slovakia and the 
Czech Republic 4-year and 8-year grammar schools, specifically students in the 16 to 
19 year age range, and in Belgium on general secondary schools (ASO) representing an 
equivalent to grammar schools in Slovakia and the Czech Republic. 

The school subject informatics taught at most of schools at the secondary level of 
education is usually taught as a compulsory subject with different number of allocated 
hours and is conceived mainly as a “general informatics education”. This means that 
in informatics lessons students acquire basic knowledge and skills to work with digital 
technologies to search, process and present information which is necessary to solve dif¬ 
ferent tasks given them (Kalas, 2001; Stankovsky and Hauser, 2008; Hauser, 2008a; 
Hauser, 2008b). A part of the informatics subject matter can be related also to algorith- 
misation issues. In Slovakia there are five main topics included in the informatics cur¬ 
ricula at secondary schools. These are: 

• Information around us. 

• Communication by the means of ICT. 

• Problem solving and algorithmic thinking. 

• Basic principles of ICT operation. 

• Information society. 

Like in Slovakia, teaching informatics in the Czech Republic is carried out as well as 
a kind of “information and communication education” with the goal to prepare students 
to find, verify, use, interpret and present information in various forms and to develop 
students' critical appreciation of information and making informed judgment as users of 
information (Kvasna, 2007). The main topics included in the informatics curricula in the 
Czech Republic are similar to the above-mentioned in Slovakia: 

• Informatics, information. 

• Computer as a mean for the work with information. 

• Internet, communication, computer networks. 

• Text editors, text processing. 

• Presentation technologies. 

• Spreadsheets. 

• Database systems. 

• Additional recommended topics. 

In Belgium, from the content standards' point of view, it is the curriculum of infor¬ 
matics taught at the general secondary schools (ASO) which are representing an equiva¬ 
lent content to the mentioned one taught at grammar schools in SK and CZ. The subject 
content is structured into following 5 topics: 

• Computer technology and computer networks. 

• Familiarization with a selected software environment (development of key com¬ 
petencies; system software; application software - text editor, spreadsheets, pre¬ 
sentation and database software). 

• Communication through ICT. 

• Solutions in simple development environments. 

• Social and ethical aspects of ICT. 
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Unlike informatics, programming as a separate school subject is not taught in each 
school, what is e.g. the kind of the general secondary schools (ASO) in Belgium. If pro¬ 
gramming is not taught as a separate school subject, some topics can be included in the 
informatics curriculum. Usually in such a case the number of lessons allocated to these 
topics is very small, and they (i.e. algorithmisation and programming) are taught only 
on a very informative level. For example in Slovakia the range of subject matter devoted 
to algorithmisation and programming within the compulsory subject informatics is in¬ 
cluded in the topic Problem solving and algorithmic thinking. 

If programming is taught as a separate school subject it can be taught as a compulso¬ 
ry subject but very often it is offered to students as an optional one or in a form of several 
optional alternative subjects. The curricula content in these subjects use to be focused on 
teaching theory of formalized languages and their grammars, theory of automates and 
their operation systems, and mainly on acquisition knowledge and skills in program¬ 
ming in a particular programming language. As the most suitable language, facilitating 
structured programming, Pascal was taught in many secondary schools in Slovakia in 
the past. This alternative was one of the most effective, but the environment which has 
still been used in some schools presents already an obsolete DOS development environ¬ 
ment Turbo/Borland/Free Pascal based on text user interface. Currently the trends are to 
make students familiar with event ruled programming, which they use in algorithmisa¬ 
tion as well as in acquiring knowledge and skills in programming methods. Following 
these trends Object Pascal, and on it based development environment Borland Delphi, 
is more and more taught programming language within programming lessons at upper 
secondary schools (Capay and Magdin, 2012). It is also due to the fact that Object Pas¬ 
cal is a natural Pascal successor and moreover currently it presents the most elaborated 
and probably for school conditions the most suitable RAD tool to create applications in 
Microsoft Windows environment. Avery similar situation is also in the Czech Republic. 
But unlike Slovakia, in the Czech Republic some schools leave Delphi environment 
because of license and financial reasons and they move to Lazarus and FreePascal or the 
programming language C. 

As it has already been mentioned the whole research was focused on assessment of 
selected aspects of both school subjects related to computer sciences, informatics and 
programming. Both parts of the research were carried out at the same time by the same 
methodology, using the same research samples and data collection questionnaire (in 
one administration of the questionnaire its items asked on the school subject informat¬ 
ics and aspects of its teaching, and in the second case the respondents expressed their 
opinions again to the same items but this time in relation to the subject programming 
and its teaching). 

Relevant research results obtained for the school subject informatics taught in Slova¬ 
kia, the Czech Republic and Belgium we published in Informatics in Education (Zahorec 
et al, 2012). In this paper we present main findings obtained for the subject program¬ 
ming and its teaching in Slovakia and the Czech Republic. Belgium is not involved 
here, as in the observed (ASO) type of the secondary schools in Belgium, relevant to the 
observed Slovak and Czech grammar schools, programming is not taught independently 
as a separate school subject. Additionally to the main results obtained for the subject 
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programming, we present also their comparison with the results of the assessment of the 
same aspects of informatics teaching (see in the part Conclusion). 


2. Research Methodology 

The quality of teaching in a subject is usually assessed through analysing its content, the 
students’ achievements, and the level of teaching. We decided to assess the current state 
of teaching programming from the perspective of the students, based on their opinions. 
Therefore, we tried to determine how certain factors which influence and determine the 
quality of teaching programming are viewed by students. Specifically, we took these 14 
factors into account: 

PI - the popularity of a subject (in meaning, whether it is a favourite subject). 

P2 - the applicability of gained knowledge in the students’ future. 

P3 - attractiveness of the curriculum content. 

P4 - the demands of the curriculum. 

P5 - the clarity of presentation of new material. 

P6 - the attractiveness of curriculum presentation by teachers. 

P7 - the suitability of particular methods used by teachers for curriculum presenta¬ 
tion. 

P8 - the engagement level of tasks to be solved. 

P9 - the clarity of textbooks used. 

P10 - the usability of knowledge for solving practical problems. 

Pll - the attractiveness of teaching aids used. 

P12 - the way in which students make written notes of the presented subject matter. 

P13 - the appropriateness of specific methods in written notes preparation for stu¬ 
dents. 

P14 - sources of concern related to the subject. 

These factors were found important in our previous experience and research results, 
in professional literature in this area (Kurland et al., 1989; Holmboe etal., 2001; Lapidot 
and Hazzan, 2003; Ragonis et al, 2010; Van Diepen et al., 2011; Saeli et al., 2012) and 
in communications with other experts (both research workers and informatics and pro¬ 
gramming teachers from various types of schools and with various durations of teaching 
experience). 

Data collection was done through a questionnaire, in which the individual question¬ 
naire items corresponded to the aforementioned 14 factors and in which students ex¬ 
pressed their views and opinions on this factors. 

Research samples in the observed countries (Slovakia - SK, Czech Republic - CZ) 
were, due to our limited opportunities, based on the availability of schools. The schools 
from both countries involved in the research, however, represented different regions as 
well as different sizes of towns. The total number of respondents was 238; 177 respon¬ 
dents were from Slovakia and 61 respondents from the Czech Republic. The Slovak 
respondents were from 8 different grammar schools and the Czech respondents repre¬ 
sented 6 grammar schools. The schools were selected from different towns, both bigger 
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and smaller, located in different regions either of Slovakia or the Czech Republic, and 
they varied also in their size (numbers of students attending the schools). With regard 
to the sample coverage of students and their study profiles, it is important to note that in 
both groups (SK, CZ), students were attending classes with a curricular focus on infor¬ 
matics or programming. 


3. Data Collection 

We based the evaluation of the factors, which affect the quality and attractiveness of 
programming teaching in schools, on the students’ assessments. We are aware that due 
to this fact (i.e. the lack of evaluation of programming education by teachers as well as 
the evaluation of teachers by other observers), our findings cannot be taken as general. 
However, we consider students’ reviews important and telling in the context of identify¬ 
ing the strengths and weaknesses of any subject. 

Based on what has already been stated, students evaluated programming both in 
terms of their subjective relationship to the subject (for example, the popularity of the 
subject, attractiveness of the content, future application of the subject) and also in terms 
of their view of the implementation of the subject’s curriculum (for example, clarity of 
textbooks used, method of interpretation/presentation of the curriculum by teachers, use 
of teaching aids in classroom, attractiveness of used teaching aids). Relevant attitudes 
and opinions of students were surveyed in a questionnaire in which the individual ques¬ 
tionnaire items corresponded to the aforementioned 14 factors. 

To the questionnaire items PI, P2, P3, P4, P5, P6, P8, P9 and P10 respondents ex¬ 
pressed their answers (evaluations of the relevant assessed factors) according using 
a 7-point scale (1 - most negative assessment/attitude; 4 - neutral, emotionally indiffer¬ 
ent assessment/attitude; 7 - most positive assessment/attitude). 

In the questionnaire items P7 and P11-P14, the respondents were asked to choose 
among several given alternatives the most suitable answer for them. The offered alterna¬ 
tives to the particular items were the following: 

P7 - the kind of explanation preferred by the student: 

a) teacher explains the subject matter without using teaching aids ; 

b) teacher explains the subject matter using various teaching aids; 

c) teacher involves also students in the explanation of the new subject matter, 

d) teacher gives individual tasks to students and supervises their progress', 

e) if other, state what you like. 

PI 1 - positive or negative answer to the question whether there are also other teach¬ 
ing aids used during the lessons of programming in addition to computers (PI la) and in 
case of a positive answer, the assessment of how interesting the used teaching aids are 
for the students (PI lb): 

a) very uninteresting', 

b) uninteresting', 

c) rather uninteresting', 

d) neither uninteresting nor interesting'. 


162 


J. Zahorec, A. Haskova. M. Munk 


e) rather interesting', 

f) interesting', 

g) very interesting. 

P12 - the way in which students are asked to make their written notes: 

a) the teacher dictates us the notes', 

b) we do our written notes according to the teacher who writes the notes on the 
blackboard or presents electronic notes using a data projector, 

c) a part of the written notes we do according to the notes made by the teacher and 

a part of the notes we do from the textbooks ourselves', 

d) we make our written notes completely by ourselves on the basis of the teacher‘s 

explanation', 

e) we make all our written notes from the textbook ourselves at school', 

f) we make all our written notes from the textbook ourselves at home', 

g) we do not make written notes of the presented subject matter at all. 

P13 - positive or negative answer to the question of whether note taking method 
is convenient for the student (PI3a) and in case of a negative answer indicating the 
one, from the same alternatives as stated in PI2, which the student would prefer more 
(PI 3b). 

P14 - possible reasons for feeling scared or nervous before programming lessons: 

a) I am not afraid of anything', 

b) unpreparedness/I am not prepared properly, 

c) oral examination', 

d) practical tests', 

e) getting a bad grade', 

f) fear of repeated lack of understanding the presented subject matter, 

g) other, state what. 


4. Data Processing 

The questionnaire reliability was proved through identification of its suspicious items by 
the means of the analysis of the questionnaire/item analysis. The final total questionnaire 
reliability a = 0.84, calculated using the Cronbach alpha, indicates a high inner consis¬ 
tence of this measuring tool as a tool for relevant reliable research data collection. 

Data obtained for each questionnaire item were processed to test the effect of the fac¬ 
tor COUNTRY. Statistical processing of the collected data was based on cA/'-square tests, 
calculating the contingency coefficient, analysis of variance for repeated measurements 
ANOVA, Greenhous-Geisser and Huynh-Feldt corrections for repeated measures in the 
analysis of variance, and on graphical visualization. 

The analysis of variance for repeated measures (RM-ANOVA) was used in tests of 
differences between two or more dependent samples - repeated measures. RM-ANOVA 
was used when, apart from the factor which determines the repeated measure in particu¬ 
lar objects - dependent samples, such as items P1-P6, P8-P10, there are also factors 
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such as COUNTRY which divide the objects into several groups - independent samples. 
If the independent samples are compared, this factor is called a between-groups factor 
(COUNTRY). If the dependent samples are compared - repeated measures of the same 
objects - the factor is called a within-group factor (ITEM). Calculations of the tests of 
significance are different for these different types of factors but the interpretation of their 
results is the same. 

To be able to use the tests, following assumptions have to be satisfied: 

• Normal distribution of the dependent variable in the groups according the factor 


levels. 


• Assumption of the covariant matrix assumption of sphericity - equal variances 
and covariances in the covariant matrix for the repeated measures. 

To find out relations between two nominal variables, analysis based on contingency 
tables was used. Hence, the contingence analysis was used to analyse nominal items 
dependences (P7, Plla, P12, P13a, P14 x COUNTRY). The group of the non-parametric 
tests belonging to the contingency analysis is based on the contingency table. This analy¬ 
sis tests the null hypothesis, which states that the variables are independent opposite to 
the alternative that variables are dependent. 

For the analysis of tables with multiple fields we used the c7z/-square test of indepen¬ 
dence. The chi -square test of independence represents an enhancement of the chi -square 
test of goodness of fit and results from the contingency table of the observed frequencies, 
where the observed frequency a is the frequency of the combination x. A y . 

The expected frequencies are those that fulfil the null hypothesis of the independence 
of two variables. The expected frequency of the relevant field is equal to the product of 
the relevant observed frequency of the line and column divided by the total number of 
the observations. 


The c7?;-square test verifies whether the differences between the observed and ex¬ 
pected frequencies are random (the variables are independent) or statistically significant 
(the variables are dependent). 



( 2 ) 


The cA/'-square test can be used only in the case when the expected frequencies are 
sufficiently large. In cases when the expected frequencies are not big enough, the Fisher 
test can be used, but this is applicable only for tables with four cells. 

To assess the degree of dependence between two nominal variables the contingency 
coefficients (Pearson, Cramer V) were used. These represent values from the interval 
<0, 1>, where 0 means no relation and 1 means an ideal relation. Their significance was 
tested by the means of the chi -square test of independence. 
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More detailed information on the conceptual and methodological bases of the research 
and the pilot verification of the administered questionnaire reliability was published in 
(Zahorec and Haskova, 2011). 


5. Analysis and Interpretation of Research Results 

5.1. Analysis and Interpretation of the Results of the Ordinary Items 

The plot of averages and confidence intervals (Fig. 1) illustrates the results of the evalua¬ 
tion of questionnaire items PI to P6 and P8 to P10 as a whole, dependant on the COUN¬ 
TRY factor. From the graph, we can see a tendency for Slovak students to assess program¬ 
ming teaching more positively than the Czech students. Slovak respondents achieved an 
overall score (mean score for all monitored items) of 4.63. Czech respondents achieved 
a slightly lower overall average of 4.39. Interval estimates of the average responses of 
the two groups of respondents overlap, thus confirming the statistical independence of 
the respondents’ answers to the questionnaire items as a whole from the COUNTRY 
factor (as confirmed by the results of multidimensional analysis of variance for repeated 
measures presented in Table 1; note the achieved value of p > 0.05). On the other hand, 
we observed that, in terms of this statistical indicator of answers to each monitored factor 
(questionnaire items), there was greater variability in the group of Czech students than 
among Slovak students. 

After investigating the overall score of the questionnaire (items PI, P2, P3, P4, P5, 
P6, P8, P9 and P10), we tested which items have significant differences in the responses 
in terms of the COUNTRY factor. 


Programming 



STATE 


Fig. 1. Averages and confidence intervals of scores in questionnaire items 
according to the COUNTRY factor. 
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Table 1 

Multidimensional analysis of variance for repeated measurements 



Sum of Squares 

Degrees of freedom 

Mean Square 

F-statistic 

p-value 

Absolute value 

33177.90 

i 

33177.90 

3302.846 

0.000000 

COUNTRY 

23.23 

i 

23.23 

2.312 

0.129698 

Error 

2370.68 

236 

10.05 




An assumption for the repeated measures ANOVA is the equality of variance and 
covariance in the covariance matrix of repeated measures. This assumption is called the 
covariance matrix sphericity condition. If the condition of sphericity is not satisfied, type 
I error increases. In such cases, degrees of freedom used for the F-test are adjusted using 
certain corrections, thus achieving the declared level of significance. 

Due to the violation of the conditions of sphericity of the covariance matrix, we used 
the Greenhouse-Geisser and Fluynh-Feldt correction (Lower Bound) for repeated mea¬ 
sure analysis of variance to find the statistical significance of differences in the responses 
of the respondents (Table 2). 

The test results of the respondents’ answers to items PI to P6 and P8 to P10, accord¬ 
ing to the Greenhouse-Geisser and Huynh-Feldt-correction (Lower Bound) for repeated 
measures of variance analysis (Table 2) confirmed the statistical significance (p < 0.01) 
of differences between item ratings. On the other hand, differences between responses to 
the questionnaire items, depending on the nationality of the respondents (COUNTRY), 
were not significant (p = 0.121 > 0.05). 

Table 3 presents the results of the identification of homogeneous groups in the evalu¬ 
ation of individual items without differentiation into groups according to the COUN¬ 
TRY factor, but depending on the factors observed (questionnaire items). Flomogeneous 
groups are obtained through multiple comparisons of pairs. 

The respondents in the three identified homogenous groups responded the items of 
the correspondent group almost identically regardless of their nationality. There was no 
confirmed statistical significance in the differences of the responses to the items within 
each homogeneous group. 

Test results confirmed the significance of the P4 and P9 items (third homogeneous 
group), in which there were statistically significant differences in the evaluation of stu¬ 
dents’ responses in comparison with all other tested items. We visualize this observation 


Table 2 

Greenhouse-Geisser and Huynh-Feldt correction (Lower Bound) 
for repeated measures of variance analysis 



Lowr.Bnd 

Lowr.Bnd 

Lowr.Bnd 

Lowr.Bnd 


Epsilon 

Upr.svl 

Upr.sv2 

Upr.p 

ITEM 

0.125000 

1.000000 

236.0000 

0.000001 

ITEM*COUNTRY 

0.125000 

1.000000 

236.0000 

0.120666 
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Table 3 

Identification of homogenous groups 


ITEM 

Average 

1 

2 

3 

P4 

3.823529 




P9 

3.869748 




P8 

4.432773 


**** 


P2 

4.710084 


**** 


P6 

4.760504 


**** 


Pi 

4.815126 

**** 



P10 

4.827731 

**** 



P3 

4.903361 




P5 

4.945378 

**** 




in Fig. 2, where the average interval estimates overlap for these two items while they do 
not overlap with the remaining items. 

The second identified homogenous group (P8, P2 and P6) overlaps with the first ho¬ 
mogenous group (P2, P6, PI, P10, P3 and P5) at the level of items P2 and P6. Also, from 
the data in Table 1, we can see a noticeable shift in the positive evaluation of the items in 
the first homogeneous group compared to the items in the third homogeneous group. 

The results of the statistical analysis for the assessment of selected factors affecting 
the quality and attractiveness of programming classes by students (ordinary question¬ 
naire items PI to P6 and P8 to P10), obtained separately for each of the questionnaire 
items (individual factors) without differentiation into groups according to the national¬ 
ity of the respondents (based on statistically non-significant differences between the 
responses of Czech and Slovak respondents), are presented in Fig. 2. This graph shows 
the point and interval estimates of the average ratings of individual items. 


Programming 



Fig. 2. Visualisation of differences in the evaluation of individual ordinary items. 
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As can be seen in the data recorded on the graph (Fig. 2), the highest total average 
scores were recorded in items P3 and P5 (see also data in Table 3). The highest student 
rating was thus achieved in the factor of attractiveness of the content of the curriculum 
(5 points means the content is more attractive rather than unattractive ) and the factor of 
clarity of the teacher’s presentation of new curriculum (also scored 5 points on the scale, 
i.e. the evaluation of teacher’s presentation was I understand more often than not). The 
lowest overall average score was recorded for items P4 and P9, where the averages are 
significantly lower than the evaluations obtained for other items. These items assessed 
the difficulty of the curriculum (P4) and clarity of textbooks used (P9). In both cases, 
the outcome of the evaluation was “neutral” (score 4 on the scale), which, in the case of 
subject-matter assessment, means neither difficult nor easy and in the case of the text¬ 
books, neither satisfactory’ nor unsatisfactory. 

Despite the fact that the differences between the groups of Czech and Slovak respon¬ 
dents are not statistically significant, a comparison of the statistical processing of the 
responses to each item, when taking into account the COUNTRY factor (see graph in 
Fig. 3), shows a tendency for Slovak respondents to assess individual factors in a more 
positive way than the Czech respondents. Overall, however, the observed differences are 
very small and the response curves for the two groups are similar to each other, which 
also confirms the independence of responses from the COUNTRY factor. The graph in 
Fig. 3 also shows that the results for the overall indicator of variability of responses to 
individual items reached approximately equal values in the two groups. 

Czech students graded the attractiveness of the programming lesson content the 
highest (P3 mean score 5.1, with a score of 5 meaning the content of the course was 
rather interesting, 4 - neither interesting nor uninteresting and 6 - interesting). Item P3 
achieved the highest average score in the responses of all respondents (see Fig. 2), but 
achieved a slightly lower rating by the Slovak respondents (4.8). It might be appropriate 
to think about how to reach students better and help them develop a deeper interest in 


Programming 
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Fig. 3. Average point and interval score of each item, divided by the COUNTRY factor. 
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the subject, particularly with regard to their age group. It should be noted, however, that 
each person can interpret the word interesting differently. For students in their final year 
of grammar school, alternative, non-standard algorithmic procedures in problem solving 
may seem interesting, while beginners and students who are confronted with this issue 
for the first time might find the decomposition of a problem and the recording of indi¬ 
vidual commands interesting. 

In addition to the attractiveness of the curriculum, Czech students highly rated the 
importance of the knowledge gained in the study of algorithms and programming for 
their personal interests and future professional functioning in society (P2 = 4.8, i.e. eval¬ 
uation reaching a score of 5 on the scale used, which means a rating of I will probably 
use the knowledge gained). They also declared their programming class as rather popular 
(PI = 4.6, i.e. closer to rather popular than to neither popular nor unpopular, within the 
range used). 

An interesting result in the case of the Czech respondents is the highest value of 
variability (1.75) in item P2, which means the greatest diversity in the statements of the 
respondents in the examined factor (utilization of knowledge in their own future). 

The overall assessment of programming education as rather popular by the Slovak 
(PI = 4.9) and Czech (PI = 4.6) groups of respondents can be considered as highly posi¬ 
tive when seen in the context of domestic and foreign research (Bilek, 2008; Hofer and 
Svoboda, 2005; Bilek et al, 2006; Dopita and Grecmanova, 2008). This research found 
overall trends to reduce the popularity of natural science subjects in today’s youth (not 
only in Slovakia and the Czech Republic, but also in other EU countries) as well as a 
constantly declining interest in pursuing studies in science and engineering. However, 
this result can be related to the academic focus of the respondents in both groups (their 
study programmes), since we analysed students in either 8-year grammar schools in 
classes focused on programming, or students in a 4-year grammar school with extended 
informatics classes. The overall assessment of students’ relationship to the subject may 
be influenced by the study program in the grammar schools that puts a greater emphasis 
on informatics, programming or mathematics, which, in addition to the hourly load of 
the course, may influence also the level and the clarity of teaching instruction (see the 
average scores of the item clarity of teacher’s presentation of new curriculum P5 = 5.1 
and the attractiveness of curriculum presentation P6 = 4.8 in the group of the Slovak 
respondents). Indirectly, this result may reflect also the overall level of professional and 
career training for future informatics and programming teachers provided by the univer¬ 
sities in Slovakia and the Czech Republic (Gergelova et al., 2009). 

The achieved results in the P3 items of both groups of respondents indicate how 
important it is to include current problems in the curriculum. Students then see that they 
are expected to learn something meaningful for their future life and not just for a grade in 
school. In teaching programming, the most important aspect is the selection of a modern 
platform, programming language, and the associated development environment. Stu¬ 
dents need to be aware that the course does not teach them something obsolete, but that 
they are learning to program in the language and development environment that are ap¬ 
plicable in their possible future professional direction. Modern platforms with rich and 
easy-to-use libraries of classes contribute to the overall attractiveness of programming 
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from the perspective of students. The selection of a modern platform and a language, 
however, is not the only thing that makes the study of programming attractive. The in¬ 
clusions of practical and interesting tasks in the curriculum, i.e. tasks that allow students 
using their creative potential, also increase the attractiveness of programming classes. 
Setting programming tasks that are useful for students and exciting in their application 
is the means by which students can express their creativity, develop their ability to solve 
more challenging tasks, and become motivated to their further self-education. Offering 
students the opportunity of programming tasks, such as selecting all prime numbers 
from a randomly generated list, finding the greatest common denominator of two given 
numbers, or solving quadratic equations, will not appeal to students very much. We be¬ 
lieve that interesting tasks involve the programming of interactive multimedia applica¬ 
tions or even simple games. 

We consider the average score in item P5 in the Slovak group of respondents as 
a positive result in regards to the quality of programming teaching. The value of 5.1 
means that students grade the clarity of the teachers’ presentations of new curriculum as 
essentially positive, because this value represents an assessment of rather understand¬ 
ing at our scale. Of course, this depends on the knowledge that was attained in preced¬ 
ing classes, and its links to the current curriculum. Nevertheless, practical and applied 
aspects of programming education should not be forgotten. Therefore, we consider it 
important that during lessons, teachers do not present the subject matter in an artificial 
way, but rather through interesting application presentations, which thoroughly explain 
the essence of the problem and its algorithmic implementation in the relevant source 
code (programming language). For example, the explanation of programming structures 
should not be limited only to the presentation of general command syntax and notation. 
It is important to try and find some interesting application problems that can be used 
in explaining the curriculum. Students then have the opportunity to learn not only the 
algorithmic structures themselves, but also their real significance and impact in solving 
specific problems. 

The lowest average score, which was significantly lower compared to the other items, 
was the same in both groups - item P4 (the demands of the curriculum) and P9 (clarity 
of textbooks used). 

Czech students consider the curriculum in programming to be rather difficult 
(P4 = 3.4), while Slovak students evaluated it as neither difficult nor easy (P4 = 4.0). 
This raises an interesting aspect of our findings. It is the fact that the Slovak respondents 
generally, on the one hand, characterize programming as a subject they consider to be 
rather popular (PI =4.9) and its content interesting (P3 =4.8) and teachers’ presenta¬ 
tion of curriculum as basically easy to follow (P5 = 5.1, i.e. a score of 5 - the teacher’s 
interpretation of the new curriculum is rather understandable), but on the other hand, 
they declare an essentially neutral stance on the complexity and difficulty of the subject 
(P4 = 4.0, i.e., a score of neither difficult nor easy). Of course, it is possible that the 
evaluations of items PI, P3 and P5 are heavily influenced simply by the “moderate dif¬ 
ficulty level” of the programming curriculum (a score of neither difficult nor easy). An 
analogous situation can be observed in the group of Czech respondents. The results of 
the research, on the one hand, suggest a close relationship between the evaluation of the 
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attractiveness of the subject content (P3) and the popularity of the subject (PI), but, on 
the other hand, do not support the conclusion that students consider favorite subjects to 
be explicitly easy. It is assumed that the basis for mastering the curriculum of algorithms 
and programming is logical thinking, which is associated with the practical nature of 
this subject. Programming is not about learning something by heart, but rather about 
understanding the logical essence of the problem and designing students’ own programs 
using a systematic approach. 

As for the questionnaire item P9, the group of Czech respondents assessed the text¬ 
books used in programming classes by their clarity, catchiness, and attractiveness of 
content as rather unsatisfactory (P9 = 3.4) and the Slovak respondents rated them as 
neither satisfactory nor unsatisfactory (P9 = 4.0). In the case of Slovak respondents, 
however, this item recorded the highest value of variance (1.79), which shows the great¬ 
est variability in the statements of the respondents. These wide-ranging differences of 
opinions can be attributed to the inconsistent choices and use of textbooks and supple¬ 
mentary teaching materials in various schools in Slovakia (in our case, grammar schools) 
in programming classes. 

It is possible to diagnose P4 and P9 as factors adversely affecting the quality of pro¬ 
gramming teaching. This is mainly the case of the item P9. As shown by the results of 
our research, the use of programming textbooks is rather unsatisfactory for the students 
in some aspects. This can be influenced by examples used in the textbooks that address 
only trivial, uninteresting, or unattractive problems for students. Some textbooks can be 
too academic for this age group (16-19 years). The problem of ensuring proper quality 
programming textbooks needs to be addressed by analysing the current state and sepa¬ 
rate needs for the Slovak and Czech education systems. The selected textbooks should, 
on the one hand, present students with examples based on their experience, and, on the 
other hand, explain the techniques in the programming language used for solve the task 
in a systematic way. 

5.2. Analysis and Interpretation of the Results of Nominal Items 

The null hypotheses for each nominal item were expressed in such a way that the answer 
to the relevant item does not depend on the factor COUNTRY, against the alternative 
that it depends on. Most of the null hypotheses were disproved (see the p-values smaller 
than 0.05 shown in Table 4). The differences between the respondents’ answers depend¬ 
ing on the factor COUNTRY were statistically significant in the following cases: 

• Item P7 in relation to the factor COUNTRY. 

• Item PI la in relation to the factor COUNTRY. 

• Item P12 in relation to the factor COUNTRY. 

With the P7 item we aimed at finding ways of curriculum presentation that best 
meet the needs of the students (Fig. 4). In the group of Slovak respondents, the most 
frequent response as the most appropriate method of presentation of the new curriculum 
by teachers was the choice c - teacher involves the students in the presentation of the 
new curriculum as well (45.8%), while the second most frequently recorded answer 
was b - the teacher explains the curriculum himself/herself using various teaching 
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Table 4 

The results of the chi -square test of independence for nominal items according to the COUNTRY factor 


ITEM / FACTOR 

Pearson’s c/ 2 /-square test 

Contingency 

coefficient 

Cramer 

coefficient 

x 2 

sv 

P 

X 2 

x 2 

P7 (5) / COUNTRY (2) 

13.1936 

4 

0.0104 

0.2292 

0.2354 

Plla (2) / COUNTRY (2) 

4.9161 

1 

0.0266 

0.1423 

-0.1437 

PI2 (7) /COUNTRY(2) 

15.7321 

5 

0.0077 

0.2490 

0.2571 

PI3a (2) / COUNTRY (2) 

0.4971 

1 

0.4808 

0.0457 

0.0457 

P14 (7) /COUNTRY(2) 

6.5097 

6 

0.3686 

0.1632 

0.1654 



P7 

Fig. 4. Absolute frequencies of answers a-e for the item P7 according to the COUNTRY factor. 


aids (39%). In the group of Czech respondents, the most frequent response was option 
b (63.9%) and the second most frequent response was option c (24.6%). Other options 
didn’t exceed the level of 9.0% (a - SK= 7.3%, CZ = 8.2%; d- SK = 5.7%, CZ = 3.3%; 
e - SK = 2.3%, CZ = 0.0%). 

Item PI la was designed to determine the use of other teaching aids in teaching pro¬ 
gramming (aside from computers). The vast majority of respondents (56.7% out of 238 
in total) stated that during programming lessons, teaching aids are not used. In terms 
of differentiating by the COUNTRY factor (Fig. 5), a no answer was given (we use no 
teaching aids aside from computers ) by 68.9% of Czech respondents and by 52.5% of 
Slovak respondents. The Slovak result can be viewed as a positive finding in regards to 
the level of programming teaching at Slovak schools. 

The graph in Fig. 6 shows the differences in the responses of Czech and Slovak 
students for item P12 (Which method of note-taking do you use in programming class), 
which achieved the highest level of significance of differences compared to the remain- 
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Fig. 5. Absolute frequencies of negative and positive answers for the item PI la 
according to the COUNTRY factor. 


ing contingency coefficients P7 x COUNTRY, Plla x COUNTRY, P13a x COUNTRY, 
P14 x COUNTRY (contingency index of 0.249). The results of the c/zz-square test are 
also confirmed by the response curves for the item P12 for each country that are not 
similar to each other. 

From the interaction diagram (Fig. 6) we see, that while in Slovakia the dominant 
method of note-taking is to follow teachers’ dictation, the Czech students use signifi¬ 
cantly different ways to record the curriculum content. Making individual notes based 
on the teacher’s presentation is just as frequent as when the teacher dictates notes to stu- 



Fig. 6. Absolute frequencies of answers a-g for the item P12 according to the COUNTRY factor. 
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Fig. 7. Absolute frequencies of answers a-g for the item P14 according to the COUNTRY factor. 


dents. The subsequent answers of respondents to the question PI 3a showed that the note 
taking method required by the teachers is the preferred method in both Slovak (83.1%) 
and Czech (86.9%) schools. 

In the case of item PI 4, in which we assessed what evokes a sense of fear and anxiety 
in the surveyed students before a programming class, the most frequent response for both 
groups of respondents was that they usually have nothing to fear before programming 
classes. The frequency of potential sources of fear and anxiety before programming les¬ 
sons for both Czech and Slovak respondents separately is represented on the graph in 
Fig. 7. The response curves of Czech and Slovak respondents mirror each other, which 
confirms the results of the chi -square test (p = 0.369), i.e. the absence of a statistically 
significant difference between the responses of the groups of respondents to this item. 


6. Conclusion 

As the presented research results show, the most problematic aspect of programming 
lessons in both countries is the use of textbooks. Students evaluate the used textbooks of 
programming from neutral to negative. Ensuring proper quality programming textbooks 
should therefore be the primary goal for both education communities (SK as well as CZ). 
In the development of textbooks, close attention should be paid to creating attractive and 
interesting tasks for students, since the tasks that teachers currently assign do not seem 
to interest the students (they are rated as neither interesting nor uninteresting), and thus 
students are not very motivated to solve them. 

In light of a lack of textbooks of proper quality, the professional performance level 
of teachers of programming must be highlighted. The clarity of their presentation of 
new curriculum can be identified as strength of programming teaching, mainly in the 
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monitored schools in Slovakia. Unlike the tasks solved, which the students did not rate 
as interesting, the teachers’ presentations of the curriculum in terms of attractiveness are 
evaluated by students as fairly positive ( rather interesting). 

A negative aspect of our results is the found out low usage of various teaching aids in 
programming classes. On the one hand, due to the nature of the school subject, it is logi¬ 
cal that besides own computers, no other teaching aids are used. But on the other hand, 
inclusion of teaching aids is just one of the possible ways for increasing the attractive¬ 
ness of classes and students’ interest in the assigned tasks and in solving the problems 
presented to them. Despite the overall low frequency of the use of various teaching aids 
in programming education in Slovak and Czech schools, the situation in Slovakia can 
be evaluated as much more favorable, because the results showed a significantly greater 
rate of the use of teaching aids by Slovak teachers compared to Czech ones. 

While the use of teaching aids is rated more positively in Slovakia, the opposite is 
true when considering the factor of note-taking of new curriculum. In the Czech Repub¬ 
lic, students are, in a significant way, led to take their own notes in addition to copying 
what the teacher dictates. By contrast, the preferred and dominant note-taking method in 
Slovakia is for the teachers to dictate the exact notes the students should take, which we 
consider to be a weakness of programming teaching in Slovakia. To a large extent, this 
may be due to the lack of quality textbooks. On the other hand, the textbook problem has 
also been identified in the Czech Republic, where teachers, in spite of this, do not forget 
to develop students’ independence even while taking notes. 

A gratifying result is the fact that students (both Czech and Slovak) assessed pro¬ 
gramming as a subject, in which the content is rather interesting, and the knowledge 
they acquired is considered to be rather necessaiy in solving practical tasks in the future. 
They also assume that this knowledge is rather usable in their future day-to-day lives. 

Comparing the achieved results for the school subject programming with those ob¬ 
tained for the subject informatics (Zahorec et ah, 2012), it can be stated that they are 
very similar, or in a close approximation one can said that they are the same. 

In both cases, to the weakest aspects belong the textbooks. Either the informatics 
or programming textbooks, they were assessed at average on the level neither satisfac¬ 
tory nor unsatisfactory (value 4 of the used scale) or even rather unsatisfactory (value 
3 of the used scale) equally in Slovakia and the Czech Republic (informatics textbooks: 
SK - 4.1, CZ - 3.5; programming textbooks: SK - 4.0; CZ - 3.4). A little bit better, but 
really only a little bit, is the situation with the quality of informatics textbooks used in 
Belgium (achieved average score 4.3 at the meaning of the value 5 - rather satisfactory, 
but with surprisingly very different assessment of the female and male respondents: F - 
3.8, M - 4.8). On the other hand, a very interesting is the fact that at the item focused on 
the textbooks in case of both subjects informatics and programming, as well as in case 
of all concerned countries, the respondents’ answers are characterised by the greatest 
variance, contrary to the final values of the standard deviations of the respondent groups 
for the rest of the items which do not differ markedly. This result shows how difficult it 
is to create a textbook of computer sciences, unlike other school subjects, which would 
be suitable in general for most of the students, meeting their different interests, needs, 
requirements and especially learning styles. Having in mind the application character of 
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computer sciences, it is disputable to speak about the traditional learning styles and from 
them derived ways of new subject matter explanation. The mentioned facts do reflect 
also in aspects of attractiveness of the teachers’ presentation of the subject matter and 
capturing the students’ interest in it. So as it is difficult to create a suitable textbook ei¬ 
ther of informatics of programming so difficult it is also for a teacher to prepare a lesson 
of adequate quality (mainly not having appropriate textbook). And as the both results 
for informatics and programming show, this has nothing in common with the mastery of 
teachers. Here we have in mind the different results of assessments of teachers in aspects 
of clarity and attractiveness of their teaching performance. 

In case of informatics in the Czech Republic the attractiveness of curriculum presen¬ 
tation by teachers was assessed significantly worse than the quality of the used textbooks 
(3.3). The obtained results for programming were more positive (4.5). On the other hand 
the clarity of the teachers’ presentation of the subject matter in programming the Czech 
respondents assessed this time a little bit lower (INF - 5.5, PROG - 4.6). The Slovak 
respondents assessed either attractiveness or clarity of the curriculum presentation by 
teachers approximately equally for the both subjects. 

A significant shift was recorded as to the assessment of the demands of the subject cur¬ 
riculum. While informatics demands were assessed by students as neither demanding nor 
easy or rather easy (average scores between the values 4-5: SK - 4.2, CZ and BE - 4.6), 
programming curricula were assessed, mainly by the Czech respondents, as more de¬ 
manding (SK - 4.0, CZ - 3.4, at the scale value 3 - rather demanding). On one hand this 
result, in regard to the difficulty of the both subjects, one can logically expected. On the 
other hand, we consider as a very positive result that despite this by the respondents de¬ 
clared different difficulty of the both subjects, the respondents have more or less equally 
positive attitude to them (SK: INF - 5.3, PROG - 4.9; CZ: INF - 4.7, PROG - 4.6 at 
the scale points 4 - neither popular nor unpopular, 5 - more or less popular, 6 - favou¬ 
rite/popular). This shows that difficulty of a subject does not result in its unpopularity 
automatically. Of course, we are aware of the fact that the achieved high values of the 
average score of the subject popularity is influenced also by the fact that the respondents 
were attending classes with a curricular focus on informatics or programming where one 
can expect students with a higher interest in information and communication technolo¬ 
gies and computer sciences. 

As to the use of various teaching aids in programming education, the very low fre¬ 
quency of their use, with a more positive result for the Slovak Republic, was recorded 
also in case of informatics education. However in Slovakia the frequency of teaching 
aids utilized in teaching programming is on the same level as in teaching informatics, 
the situation in the Czech Republic in case of programming is even worse, i.e. to sup¬ 
port teaching programming teaching aids are used at schools even less than to support 
teaching informatics. On the other hand a positive finding is a shift in assessment of the 
used teaching aids. The respondents assessed those used in teaching programming more 
positively than those which teachers used in informatics teaching. 

As informatics represents a subject of a more general nature, not so “professionally 
specific”, logically one can expect a higher level of applicability within its teaching ac¬ 
quired knowledge in common life as the knowledge acquired within programming. Also 
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students attending the classes with a curricular focus on informatics or programming 
are aware of this fact, or may be just these students are aware of it, what is reflected in 
significantly different assessments given by them to the applicability of the correspond¬ 
ing knowledge in their future. The average score of this item in case of informatics was 
the same for both Czech and Slovak groups of respondents equal the value 5.3 and in 
case of programming it was only 4.8 (CZ) or 4.7 (SK). These values show that in case 
of programming the students connect this school subject with their future professional 
orientation, about which not all of them have already a clear vision (4 - hard to judge 
whether I will use the acquired knowledge). In case of informatics, as the knowledge is 
more general and presents key competencies necessary for everyday life either in infor¬ 
mation or knowledge society, the students are much surer that the acquired knowledge 
is useful for them and they will apply it in various spheres of their life (5-1 will likely 
use the acquired knowledge). 
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Programavimo mokymo Slovakijoje ir Cekijoje kai kuriq aspektq 
vertinimas 

Jan ZAHOREC, Alena HASKOVA, Michal MUNK 

Straipsnio autoriai atliko tarptautinj tyrimp siekdami jvertinti programavimo mokymp aukstes- 
nese viduriniojo ugdymo mokyklos klasese - ISCED 3 A. Buvo vertinami pagrindiniai kompiuteri- 
jos mokymo dalykai: informatika ir programavimas. Vertinimui buvo renkami sie duomenys: mo- 
kinii} vertinimai, jp atsiliepimai, nuomones. Sip duomenp parinkimas vertinimui isskiria sj tyrimp 
is kitp panasip tyrimp. Straipsnyje pristatomi gauti programavimo mokymo Slovakijos ir Cekijos 
vidurinese mokyklose tyrimo rezultatai. Pagrindines problemos, nustatytos sip salip mokyklose, 
yra neadekvatus vadoveliai, nepakankama jp kokybe, mokymuisi tinkamp knygp stoka. Pagrindinis 
programavimo mokymo Cekijos mokyklose privalumas yra mokykloje mokomo turinio patrauklu- 
mas, o Slovakijoje - aiskus mokymo turinio isdestymas. 



